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(54) Variable resolution view-tracking display 

(57) A panoramic display apparatus 1 0 mounted, at least in part on a helmet 14 to be worn by an operator, in which 
apparatus there is a projection means 16 for projecting a high resolution stereo display within a small first field of view 
containing the fovea! line of sight of an operator's eyes and a low resolution display outside said first field of view. The high 
resolution first field is used to present a detailed image of the object the operator is looking directly at whilst the low 
resolution field presents a low resolution peripheral image in the areas where the operator's eyes are incapable of 
perceiving high resolution images. One or other or both displays may be projected onto an eyeshield 24 or displayed via 
one or more active matrix liquid crystal displays which may be mounted on or form part of the eyeshield 24. 

The apparatus may be used with flight or other simulators, remote control apparatus e.g. robots, surface or space 
vehicles. Alternatively the apparatus may be combined with cameras to give an aircraft pilot an "all-round" view (fig 6). 




At least one drawing originally filed was informal and the print reproduced here is taken from a later fifed formal copy. 



AN EYE- SLAVED PANORAMIC DISPLAY APPARATUS 

The present invention relates to an eye-slaved panoramic 
display system and relates particularly, but not exclusively, to 
such an apparatus for use in applications that require remote or 
synthetic visual contact with the environment such as vehicle 
control, remote, piloting;, telerobotics and simulation. 

Presently known . systems for outside-world representation 
generally employ one of three methods for image display, namely: 
direct view monitor, video projection or, head/hehnet mounted 
display (HMD). Direct view monitors, such as cathode ray tubes 
(CRT's), offer reasonably; hif^ image resolution and are a 
popular display technique, both for sensor-derived and 
computer-generated ^^images but they offer only; a comparatively 
narrow field of view. The field of view can be increased by 
increasing the screen size but this cannot be done above a 
certain diameter without reducing image resolution or refresh 
rate. This can. also be prohibitively expensive and can incur 
unacceptable size and weight penalties.. 

Video projection systems that employ' multi-screen or 
dome-jvall video projections can provide a large ^ instantaneous 
field of view which in many cases is tiruly panoramic with over 
120° of view. Such, systenis provide the viewer with valuable 
peripheral information as well as a more realistic viewing area 
and, consequently, have become very popular in the field of 
flight simulation , These systems suffer from the prob^epi of 
reduced image quality because the gain in field of ;5ciew is 



surface at the expense of image quality (resolution and 
contrast) . 

The ESPRIT projector-based systems proposed by 
Singer-Link provides a so-called area-of -interest (AOI) visual 
simulator. In this system a wide-angle low resolution image is 
projected onto a simulator dome and is' overlaid with a higher 
resolution, but smaller, eye-slaved image,* provided by a 
separate servo-driven projector. Such a system does hot lend 
itself to stereoscopic image presentation, nor " to use by multiple 
viewers. ' i . . . . 

Head or helmet mounteiH display systems are being 
developed which present a condensed CRT-t3rpe linage onto a 
head or helmet mounted' eye piece. ^ These can offer" high 
resolution and with appropriate viewing conditions, good 
luminance contrast. The Images are' normally head-slaved (and 
could also, in theory, be ey6-slaved) so that the wearer is alDle 
to guide his/her direction of giaze through a total potential held 
of view of up to 360^. In addition HMD systems can also 
provide binocular stereopsis, something which can only be 
approximated with direct- view arid. projection ' displays . 
Unfortunately the field of view provided by HMD systems is 
relatively small (eg 60** binocuTar FOV, including 40** per 
eyepiece) and increasing the field of view without loss to image 
quality is prohibitively expensive ' in ' terms of image 
generation / con version . 
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None of the above mentioned systems is capable of 
providing a suitably sharp stereoscopic image with a large 
general viewing area without impairment of image quality or 
without overstretching the image generation /conversion systems. 

It is therefore an object of the present invention to provide 
a panoramic display apparatus which mitigates the problems 
associated with the above mentioned systems. 

Accordingly the present invention provides a panoramic 
display apparatus comprising a head positipn tracking device and 
an eye position tracking device which in combination enable an 
operator's foveal line of sight to be determined, means for 
dividing an operator's field of view into a first field containing 
an operator's foveal line of sight and a second field outside said 
first field, and projection means for projecting a high resolution 
display within said first field and a low resolution display in said 
second field. 

It will be appreciated that the above mentioned apparatus 
exploits the limitations of the human eye, which is incapable of 
perceiving high resolution images outside a narrow 
(approximately 5°) cone around the foveal Line of sig:ht, and 
provides a panoramic display apparatus that gives high image 
resolution only where the operator's eye is capable of perceiving 
it and a low resolution display outside this field. Such an 
apparatus is capable of providing a suitably sharp image with a 
large viewing area without impairing image quality and without 
over-stretching the image generation/ conversion system. 

Preferably, the apparatus includes an eyeshleld of an 
operator's helmet onto which the high resolution display is 
projectable and a remote surface onto which the low resolution 
display is projectable. 




Alternatively, tlie ayytiLL-BXiLis iiiuludes an eyeshieid of an 
operator's helmet onto which the high and /or low resolution 
displays are projectable. 

Conveniently, the above mentioned remote surface may be 
the interior surface of an aircraft canopy (or some other 
workspace interior) . 

The projection means may comprise a first projection system 
for projecting the high resolution display and a^ second projection 
system for projecting the low ^ resolution display. In a 
particularly advantageous arrangement the high resolution 
display . may be projected onto the eyeshieid of an operator's 
helmet and the low resolution display is projected onto a remote 
surface such as the interior surface of an aircraft canopy. 

In a simple arrangement the first projection system may 
comprise a first projector which presents an image alternately to 
both left and right eyes of an operator thereby to give a stereo 
image. 

If just one projector is used in this way, the projection 
system may include an LCD shutter located at or near a source 
of said image and passive polaansers before an operator's eyes in 
order to present the image alternately to both left and right 
eyes . ^ 

Alternatively, the first projection system comprises first 
and second projectors locatable and operable in combination to 
present an image to both eyes of an operator in a manner which 
gives a stereo image. 
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One or more active matrix LCD arrays may also be used to 
project the high and /or low resolution displays. The array 
is /are transparent and locatable to form part or all of an 
eyeshleld of an operator's helmet or is /are mountable on said 
eyeshleld. 

Conveniently the first field of view may extend up to or 
slightly beyond an operator's foveal boundary. Gradation means 
may be provided for modifying: the resolution of the first field at 
its periphery in a manner which matches or approximates the 
reduction in image resolution due to the human eye's 
acuity/ eccentricity curve. 

Computation means may be provided for computing the 
visual field location or locations of the high resolution first field 
or fields. 

Conveniently, the first projection system may be mounted 
on a helmet to be worm by an operator and the second projection 
apparatus may be mounted remote from said helmet, thereby to 
enable the low resolution second field projection to be projected 
onto a remote screen. 

In a particularly advantageous arrangement the second field 
of view extends to form a full field of view of up to 360^ about 
an operator's X and Y axis , , thereby to allow the operator to 
see in all directions including downwards and rearwards. 

The above mentioned panoramic display apparatus may be 
used remotely to pilot a vehicle amd/or in a robot /robotic 
system, thus avoiding exposing the operator to dangerous or 
inhospitable environments. 
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Advantcigcciisly twc ot mors pcLnGrfirrii c displo-y cxijjj3.i^tu&> 
may be linked to each other to form a network of visual displays 
for use with multiple operators and different information may be 
presented to each operator's high resolution first field of view 
thereby to provide the operators with only that information they 
require in order to perform their given task. 

In a particularly convenient arrangement a helmet may be 
provided with a panoramic display apparatus as described above. 

The present invention will now be more particularly 
described by way of example only witli reference to the 
accompanying drawings in which: 

Figure 1 is a cross sectioned side elevation of an operator's 
helmet incorporating a panoramic display apparatus according to 
the present invention and an aircraft canopy. 

Figures 2 to 4 illustrate various alternative projector 
positions . 

Figure 5 is a front elevation of two operators helmets 
incorporating panoramic display apparatus according to the 
present invention together with associated computation means, 
and 

Fig 6 is a perspective view of an aircraft incorporating a 
numijer 6f features the subject of the present invention. 

Referring to Figure 1, a panoramic display apparatus 10 
according to the present invention comprises a head position 
tracking device, such as an accelerometer i2, mounted on an 
operator's helmet 14, an eye position tracking device in the form 
of, for example, an infra-red LED oculometer 15 and a projection 
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means shown at 16 . In a preferred ^embodiment the projection 
means comprises a first projection system 18 for projecting a 
high resolution stereo display within a first field of view 
containing the foveeil Line of sight of an operator's eye 20 and a 
second projection system 22 for projecting a low resolution 
display outside said first field.. It will however be appreciated 
that oiily . one projection^, sy^ . ^^^Y ^® used if suitably 

modified to be able to project both iJie Wgh, a^ low resolution 
displays.. . . . . .. 

The first projection system 18 is preferably mounted on the 
operator's helmet .14 and arranged to project the high resolution 
display onto an eyeshield 24 of the helmet .14. The second 
projection system 22 may be arranged in any one of a number of 
ways as illustrated in figures 2 to 4 inclusive. In figure 2, the 
second projection system 22 comprises a number of static 
projectors 22a, b projecting onto, for example, tiie inner surface 
of a flight simulator dome. , .Alternatively, just one or two 
wide-angle, multi-aims, ^ ^ projectors may be provided 

and mounted, for example, above the operators head as shown in 
figure 3. Figure 4 illustrates yet another arrangement in which 
static video or laser projector (s) slaved to head-position by 
means of a flexible fibre optic link to head-position by means of 
a flexible fibre optic link 13 to the helniet; the mages would 
then pass through a wide-angle projection device 22c mounted on 
the front of the. helmet 14 before being projected on the remote 
surface 15. .In a particularly advanced and compact 
arrangement, the second projection means may be arranged to 



helmet 14 alongside iJie'Mgh resolution *d^^ (see for example 
figure 1) . 

The first projection system 18 may include one or two 
projectors 18a, b. When "just one projector 18b is used the 
system is further provided with a Liquid Crystal Shutter 26 at 
the image source arid passive pdLarisers * 28 lodaLted befor-e the 
operator's eyes 20.' In operation the shiittir acts t6 d^fl^ci: the 
source image alternatively to operator's left and right e'yes in a 
' manfier'^ which, in combinatibn with the ~ passive polarisers 28, 
acts to present an operator with a. stereo high resolution image. 
When the two projectors 18 are lise'd each may be arranged to 
project tlie high resolution image to^bne or other of an operator's 
eyes thereby to provide a stereo image 1 

In a still further ' alternative Arrangement the "first 
projection system 18 is formed by"^ one or more active matrix 
Liquid crystal displays 30, 32 well known in thie art and 
therefore not described herein/' With suitable image pTOce^ 
jiist brie display may be used to present both the hi^h and the 
low resolution fields and ijie ' sedd ^ one display^ is preferably 
mounted on or forriis 'part of the eyeshiiBld 24, When "two 
displays 30, 32 are provided it will iae appreciated tliat only that 
display ' presenting the high resolution display need be provided 
on the eyesiiield. The display presenting the low resolution 
display caii be provided on or form part of the eyeshield 24 or 
can be positioned remote frem said eyeshield 24 if desired* 
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When the one or more displays. 3 Q, ,32 are mounted on, or 
form part , of the eyeshield. 24 it or they , may be of a transparent 
type thus allojwing the operator to see real images which are 
present beyond the eyeshield 24, such as,, for example, aircraft 
cockpit instruments, and the^ like or a remotely projected low 
resolution display. ^ ^ c s 

The pculpmetex: 15, \Yluch. monitors the eye position, and the 
head position sensor 12 are each linke/i to ,a computation device 
34, .yia , cables, 36 , ; 38 ,,,respeqtively* The computation cievice is 
configured to monitor . , tl^ie , outputs f rpm , the oculome^ier and 
^position sensor, and, deteiraiiie the direction of the .operator's 
foyeal Une,. of ..sight such JLh^ the high and low ciefiiii.tlon images 
may be projected relatiye thereto .., Such , an . arrangement .is well 
known- and hence forms, no part of the pres.ent. invention other 
than to the extent that it enables the present , invention to be 
operated.^- ^ -> ^ y -. . -.T' 

In. order , to exploit fully the hjnitations of _the. human ,eye, 
which is incapable of,^ perceiving high resolution images outside a 
narrow (apprqxmiately, 5^X cone ^arpund „;the, fo veal line of ^ sight, 
thev computation, device : ^34 is also .configured, to determine the 
boundary of an operator's^ P^Sh resolution, image perception a^d 
define the bpundairy or the display apparatus high resolution 
first field relative thei^to. , Preferably, the boundary, of ^ the 
first field e3?t<snds, up . to or slightly , beyond an operator's foyeal 
boundary thereby to Bdnimise the size of the high resolution 
display and , maximise the size of the low resolution display and 
hence reduce to, a mininmim the, computation power; required to 
present the overall image to the operator. It will be appreciated 
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that txiB operator's iieao'' axicl eyets wiii be constaritiy moving as 
he observes the displky^- presented' before him, consequently the 
computation means will be constantly updating the position of the 
first and second fields In order to provide the best possible 
display to the operator. Reduction in the computation power 
required to provide the overall display frees computation power 
to be 'used to speed up the image' update ^ 'rate thereby providing 
a more realistdd dveraU image; * ' ^" * " ^ ' • ^' 

In operation, the riigh rfeisbltitidn first field inay' be used to 
present vital information to" the" operator, ^ such as weapon or 
aircraft' pei*f or mance Tiif oi?matioh , together' witli ari enhanced view 
of that pbrtioh of the overall -'display' the operator is looking 
directly aLtV "''Pdrtiohs of the low resolution display may"* be 
enhanced in "order to attract the operator's eyes to potential 
targets if desired?. ' " . -t. 

Figure 6 illustrates an aircraft 40 which is provided" with a 
'numbe?r of reiiibte' cameras 42, 44, "46, 48 and'' 50 respectively set 
for forwairdi'^' dowhWaifci^" iipward and' left anti" right' viewing 
respectiveiy . ' ' THese'^^ camisras * may be linJced " into the paiioi^amic 
cUsplky^' apparsLte the "i>Llot "^bpefratdr with ' ^outside 

world "visual 'infd directidns , anciuding directioiis 'he 

would" norEoaUy b6 unable tb see*. "^It will be ^^t)preciated that a 
pilot op^rafbit of an aircraft equipped - with sitch camieras and 
display apparatus ^ will be' able td Ibdk dowti by- rotating his head 
about axis Y-Y towards the cockpit 'fldor for "example and see a 
representation of the real world' image on the other side thereof 
thus enabling him to" make visual contact with landmarks or 
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aircraft normally outside his field of view. If the pilot operator 
rotates his head about axis X-X he will be able to make visual 
contact with landmarks etc to his left and his right as well as 
behind the aircraft if a suitable remote camera if provided to 
point in this direction • The computation means may be 
configured to be selective with the outside world information it 
presents to the pilot operator, thus preventing him from being 
distracted by irrelevant detail. 

An aircraft when provided with the above mentioned 
cameras and suitably modified to enable remote operation of the 
aircraft controls to be achieved lends itself to being; remotely 
piloted by a pilot operator using a panoramic display apparatus 
10 as described above. The operator would be provided with aU 
the visual information he requires in order to fly the aircraft 40 
without being exposed to the often hostile environment the 
aircraft is flown in. 

It will be appreciated that whilst the panoramic display 
apparatus lO^^has been described with reference to use on 
aircraft it also lends itself to use with niaimed/umnanned surface 
vehicles and submersibles, robotic systems in nuclear power 
plants and teleoperatiLon in space. 
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1. A panoramic display apparatus comprising a head position 
tracking device and an eye position tracking device which in 
combination enable an operator's foveal line of sight to be 
determined, means for dividing an operator's field of view into a 
first field containing an operator's foveal line of sight and a 
second field outside said first field, and projection means for 
projecting a high resolution display within said first field and a 
low-resolution display in said second field. 

2. A panoramic display apparatus as claimed in claim 1 
including an eyeshield of an operator's helmet onto which the 
high resolution display is projectable and a remote surface onto 
which the low resolution display is projectable. 

3. A panoramic display apparatus as claimed in claim 1 
including an eyeshield of an operator's helmet onto which the 
high and /or low resolution dLsplaj^ are projectable. 

4 . A panoramic display apparatus sis claimed in claim 3 in 
which the remote surface is an interior surface of an aircraft 
canopy. 

5. A panoramic display apparatus as claimed in claim 1 or claim 
2 in which the projection means comprises a first projection 
system for projecting said high resolution display and a second 
projection system for projecting said low resolution display. 

6. A panoramic display apparatus as claimed in claim 5 in 
which the first projection system comprises a first projector 
which presents an image alternately to the left and the right eye 
of an operator thereby to give a stereo image. 



7 . A panoramic display apparatus as , claimed in claim 6 in 
which the projection system includes a LCD shutter located at or 
near a source of said image and passive polarisers before an 
operator's eyes* . , . / . { 

8. A panoramic ; display ; ap>Raratus as claimed in claim 5 in 
which the first projection ,sys]tem comprises, a first and second 

, projectors locatabla: suid^^.^ in combination .to present an 

image .tp both ^eyeSr of Naji,, pperatox^ ^ th^^ a stereo 

image..- r^^.c^; '^k:?- r:.? ^ ' 

9. A panoramic display apparatus as... claimed, in claim 1 
; including one or mp.re , active., matrix XCP .arrays , for projecting 

the highland./ or low resolution i ^ ; v 

10. A panoramic display apparatus as claimed in claim 9 in 
which the . ^ one or more active matrix: . LQ arrays is /are of a 
transparent type: and loc^table to form part o?>, all , of an eyeshield 
of an operator's helmet or is /are mountable on paid eyeshield. 

11. r;A panoramic display apparatus ,as .claimed .in any one of the 
preceding claims in.. which the first field- of . vievy: extends up to 
or slightly beyond an, operator's fp^eal^ boundary . p/ 

12 , A panoramic display apparatus as clsdmed in aiiy. one of the 
preceding claims, including graduation meians - for , modifying the 
resolution of said first field at its periphei^r in a maiiner which 
matches the reduction In, image , resolution due to the, human eye's 
acuity/ eccentricity curve, , v 

13. A panoramic display apparatus as claimed in any one of the 
preceding claims including a computation means for computing the 



visual field location or locations of the high resblution first field 
or fields. ' ' 

14- A panoramic display apparatus as claimed in any one of 
claims 3 to 11 in which the first projection system is -mounted on 
a helmet to be worn by an operator^ ahd the second projection 
apparatus is mounted remote from said Keliiiet, 

15. A panoramic display apparatus 'as'-6lidmed in any oiie of the 
preceding claims in^ Which the ^second field of view 'extends to 
form a full field of view of up to 360° about an operator's 
vertical and/or horizdntal' axis • ' ' - * ' -"^^ * 

16". A panor^amic ' display apparatus 'as ' clairhed in any ohe= of the 
above claims when used remotely^ to ^ t»iloi a vehicle andVor'^'in a 
robot/robotic syistem. ' ^ * - ' - - '-' - • 

17. Two or ^ more panoramic displdy apparatus as claimed in any 
one of the previous claims when' to form a 
network of 'vistial' displk^^ for lis e With multiple operators^ 

18. T^wo dv more pahbramic display apparatus as - claimed in claim 
17 operable such tfiat different informatibn is 'presented iii each 
operator's Mgh rfesblutibn'- first field ^ - '^ ^^ ' 

19. '' "a pahbramic 'display ajbparatus^ iiibstmitirilly ' hereb^fore 
descinbfed with reference' t^ diid a^s lllustratbd^ in K 1 to 3 
of the ac6bmi)anying drawings. ' - ■ - ' - 

201 X'*Kelmet' when provided with 'a panoramic display apparatus 
as claimed in any one of claims 1 to 18: 
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